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Abstract 

We introduce a new method for decoding short and moderate length linear block codes with 
dense parity-check matrix representations of cyclic form, termed multiple-bases belief-propagation 
(MBBP). The proposed iterative scheme makes use of the fact that a code has many structurally 
diverse parity-check matrices, capable of detecting different error patterns. We show that this 
inherent code property leads to decoding algorithms with significantly better performance when 
compared to standard BP decoding. Furthermore, we describe how to choose sets of parity-check 
matrices of cyclic form amenable for multiple-bases decoding, based on analytical studies performed 
for the binary erasure channel. For several cyclic and extended cyclic codes, the MBBP decoding 
performance can be shown to closely follow that of maximum-likelihood decoders. 

Index Terms: Algebraic Codes, Belief Propagation, Multiple-Bases Belief-Propagation Decoding, 
Stopping Sets. 

1 Introduction 

Classical algebraic codes of short block length have large minimum distance and efficient soft-decision 
decoding algorithms [1, 2, 3]. Consequently, these codes represent a good choice for low-delay ap- 
plications where high transmission reliability is required. Algebraic codes are also frequently used as 
components of product codes and parts of concatenated coding schemes. In modern storage and com- 
munication systems, low-density parity-check (LDPC) codes are most often used as the inner coding 
scheme. For this reason it is desirable to implement a soft-input soft-output decoder for algebraic codes 
as a belief-propagation (BP) algorithm. This is a reasonable choice as the decoder can handle both 
types of codes. Since algebraic block codes have dense parity-check matrices with a large number of 
short cycles [4, 5], BP decoders offer poor error-correcting performance for such codes. 

*Parts of the results were presented at the IEEE hiternational Symposium on Information Theory (ISIT), Nice, France, 
2007. The work was supported in part by the NSF grant CCF 0644427 awarded to Olgica Milenkovic. 
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The use of redundant parity-check matrices for iterative decoding schemes when signahng over the 
binary erasure channel (BEC) has been excessively studied. Several authors proposed using a high num- 
ber of redundant checks [6, 7, 8, 9] to improve the decoding performance. This type of decoding has also 
drawn the attention of researchers who studied this concept in the context of the AWGN channel. Other 
authors proposed adaptive BP algorithms [10, 11], which iteratively optimize the matrix representation 
during the decoding process. Such schemes have large implementation complexity due to the required 
matrix reduction after each iteration. The random redundant decoding (RRD) [4] algorithm does not 
require this type of processing and obtains very promising results. This is accomplished by serially al- 
tering the applied matrix representation within the decoding process. Another closely related approach 
was described in [12], where a simple simulation-based study using randomly chosen parity-check matri- 
ces of the [24, 12, 8] extended Golay code was performed. This approach offers poor performance when 
compared both to the performance of the RRD algorithm and the algorithms described in this work. 

The approach followed in this paper draws upon the prior work of the authors on BEC decoding [13] 
and introduces a novel decoding method that operates in parallel and iterative fashion on a collection 
of parity-check matrices. Using this set of decoder representations, the algorithm performs joint output 
processing in order to estimate the transmitted codeword. This output processing can occur at various 
stages of decoding and it may have various degrees of complexity. We distinguish between techniques 
where the BP algorithms run separately and compare them to schemes where the decoders are allowed to 
exchange information. In the latter case, we investigate processing of the form of metric and complexity 
selection, averaging of probabilities [14], information combining [15], as well as certain reliability-based 
schemes. 

As the different representations of the parity-check matrix form bases of the dual code, we refer to 
the proposed approaches as Multiple- Bases Belief- Propagation (MBBP) decoding. For the purpose of 
MBBP decoding, one needs to identify classes of parity-check matrices that jointly offer good decoding 
performance. In order to accomplish this task, we propose using parity-check matrix design techniques 
originally developed for the BEC but subsequently used for signaling over the AWGN channel. This 
approach leads to good performance results, as observed both by the authors and other researchers [16]. 
Moreover, this method relies on the fact that the performance of a parity-check matrix for the BEC is 
completely characterized by combinatorial entities termed stopping sets [17]; and, that pseudocodewords 
for linear programming decoders of several classes of channels represent stopping sets for any channel 
in the Tanner graph [18, 19]. Although we focus our attention on parity-check matrix construction 
techniques for cyclic codes, the described concepts can be generalized for other classes of codes as well. 

The main differences between the existing RRD algorithm and the proposed MBBP scheme are that 
RRD operates in a serial fashion in terms of periodically permuting the received word, while MBBP 
works in a parallel manner. Further, the RRD algorithm uses message scaling processing between 
iterations that tends to increase the overall complexity of the scheme. Contrary, the MBBP algorithm 
deploys the standard update rules defined by message passing decoding within its parallel cores. Finally, 
the RRD algorithm uses a greedy search over the Tanner graphs to find a representation which offers 
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good performance as well as random shuffling techniques of the variable nodes. The MBBP algorithm 
relies on specially designed parity-check matrix families for a given code. 

The paper is organized as follows. Section 2 introduces relevant definitions and terminology, while 
Section 3 contains a description of the bases selection process. Section 4 presents a set of different 
variations of MBBP decoding algorithms. Simulation results are presented in Section 5. 

2 Definitions and Terminology 

We start by introducing the terminology related to stopping sets and the BEC. We also provide the 
terminology needed for describing the MBBP decoding approach. 

Definition 2.1. Let A be an m x n matrix, and let the columns of A be indexed by the set of integers 
J = {0, . . . ,n — 1} . For a set 1 J , we define the restriction of A to I as the m x |X| array of 
elements composed of the columns of H indexed by X. 

When transmitting over the BEC, stopping sets completely determine the failure modes of iterative 
decoders. For completeness we define these sets below [17]. 

Definition 2.2. For a given parity-check matrix H of an [n, k, d] binary linear code C, a stopping set 
S (H) of size a is a set T of a positions in the codeword for which the restriction of H to I does not 
contain rows of Hamming weight one. 

Clearly, a codeword is a stopping set and consequently the size of the smallest stopping set of any 
given parity-check matrix cannot exceed d. 

In order to compare different parity-check matrix representations with respect to their decoding 
performance, we restrict our attention to a simple evaluation criteria: the number of stopping sets of 
size less than or equal to a, for some predefined value 1 < a < d. Given a parity-check matrix H, the 
number of its stopping sets of size a will henceforth be denoted by \Sa {H)\. Although stopping sets 
are known to completely characterize the performance of iterative decoders only for the BEC, they also 
represent a partial performance indicator for transmission over the AWGN channel. This is due to the 
intimate connection between stopping sets and pseudocodewords [18, 20]. 

As we focus our attention on codes with parity-check matrices of cyclic form, a code category that 
includes cyclic codes, we also provide the following definitions. 

Definition 2.3. Let C be a binary, linear code and its dual. A parity-check matrix of C is said to 
be of cyclic form if it consists ofn — k<m<n consecutive cyclic shifts of one chosen codeword of 

. In this context, the qualifier "consecutive" implies that the {i + l)-th row of the parity-check matrix, 
1 < i < m — 1, is the cyclic right shift of the i-th row by one position. 

A code C is called cyclic if any cyclic shift of a codeword c E C is also a codeword, and it necessarily 
has at least one parity-check matrix of cyclic form. 
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For a code with at least one parity-check matrix of cychc form, we introduce the notion of a partition 
of the set of codewords of and the notion of a cychc orbit generator (cog). 

Definition 2.4. LetC be a binary, linear, cyclic code. Partition the set of codewords ofC-^ into disjoint 
orbits (subsets) consisting of cyclic shifts of one codeword. Let one designated codeword in the orbit be 
the representative of the subset. This codeword is referred to as the cyclic orbit generator (cog). 

Throughout the paper we focus our attention on cogs of minimum Hamming weight. Little technical 
modifications are required in the above definition to encompass parity-check matrices that are of cyclic 
form when restricted to a proper subset of columns, e.g. extended cyclic codes. 

Let Q be the set of cyclic orbit generators with Hamming weight equal to the minimum distance of 
the dual code, d^. A cychc orbit generator cog^ & Q, i = 1, . . . is used to construct a parity-check 
matrix Hi, £ = 1, . . . , |^|, of size mxn,n — k<m<n, such that the row-rank of the matrix is n — k^. 
This matrix consists of m consecutive right shifts of cog^. To avoid identical rows in Hi even if m = n 
holds, only cogs with a period of n, i.e. cogs with a cyclic orbit that consists of n distinct shifts, are 
considered for the construction process. 

Note that a redundant parity-check matrix of cyclic form can have at most n distinct rows^. The 
larger the value of m, the larger the hardware implementation complexity of the BP decoder. Never- 
theless, based on extensive computer simulations, it was observed that for decoding of algebraic codes 
signaled over the AWGN channel the best decoding performance is achieved for m = n. This finding 
holds for both the bit error rate (BER) and frame error rate (FER). The reason supporting this obser- 
vation is intuitively clear. Consider a parity-check matrix of cyclic form for which m = n — k, a.s shown 
in (1) for m = 3,n = 7. Here, the symbol x serves as placeholder for the bits of the generating cog of 
the matrix. 
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As can be seen from (1), not all of the seven bits participate in the same number of parity-check 
equations - the column degrees of the parity-check matrix vary with the column. There exist at least 
two bits (including the first and last) that participate in only one parity-check, and therefore have very 
low probability of being correctly decoded. Depending on the particular choice of the cog, the set of 
symbols at the beginning and at the end of the codeword is strongly restricted with respect to the 
maximum number of parity-checks it can participate in. This problem can be solved by setting m = n: 
such a row-redundancy allows for achieving equal error protection for all code symbols [22]. Therefore, 
square parity-check matrices will be used throughout the remainder of this paper. 

We conclude this section by introducing the notion of a cog family of a set of parity-check matrices. 

^Here, and throughout the paper, we only consider cogs that generate parity-check matrices with row-rank n — k. As a 
consequence, we use the word bases to describe the underlying matrices, although the considered structures are actually 
frames. Frames are over-complete systems in which every element of a vector space can be represented in a not necessarily 
unique manner [21]. 

^Cyclic matrices with m — n rows are also referred to as circulant matrices. 
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Definition 2.5. Let T\, T2, ■ ■ ■ , be a partition of the set of indices {1, . . . , i.e. 

J-i U U ■■■ U J^i. = {1, 1^1} anfi n J-/. = 0, V / ^ /', 

so that the "stopping set performance" of is fixed within each family Tf, for all I G J-/, and for all 
/ G {1, . . . , F}, and this claim is true for all families in the partition. The "stopping set performance" 
of a parity-check matrix, defined for both the BEC and AWGN channel, is the number of stopping sets 
of size up to and including d. We refer to the set {cog^}, C. G Tf, as the f-th cog family. 

3 Bases Selection for MBBP decoding 

Recall that a linear [n, k, d] code C is uniquely defined by a parity-check matrix H of rank n — k or 
a generator matrix G of rank k. There usually exists a large number of generator and parity-check 
matrices for the same code. For BP decoding over AWGN channels, one usually seeks a sparse parity- 
check matrix H. The performance of the algorithm strongly depends on some additional structural 
properties of H, such as the number and weight of pseudocodewords. 

Adding redundant rows to parity-check matrices improves the performance of iterative decoding for 
the BEC, but usually has detrimental effects on BP decoders correcting data signaled over the AWGN 
channel. This can be attributed to the increase of the number of short cycles and the density of the 
matrix. But, as was shown by the authors in [23], adding judiciously chosen redundant rows may 
increase the minimum weight of pseudocodewords (and trapping sets) of the given parity-check matrix. 

In order to exploit the benefits offered by redundant parity-check matrices with respect to pseu- 
docodeword performance, while controlling the negative effects on the cycle lengths, the following ap- 
proach can be used. Rather than decoding a received word in terms of only one parity-check matrix, one 
can use a collection of parity-check matrices, each with small row-redundancy, in parallel. The results 
of the decoders operating on different parity-check matrices can then be appropriately combined. This 
is the main idea behind MBBP decoding, and for this purpose, we propose to develop good heuristic 
techniques for identifying parity-check matrices which offer both good individual and joint decoding 
performance. 

For cyclic algebraic codes, we proved in a companion paper [13] that parity-check matrices that 
consist of cyclic shifts of carefully chosen cogs offer excellent stopping set properties [24] . In what follows, 
we focus on identifying families of cogs that obtain very good decoding performance for the AWGN 
channel. As already pointed out, the iterative decoding performance of a fixed parity-check matrix used 
over the AWGN channel is, to a certain extent, strongly correlated with its BEC performance, so that 
we use the total number of stopping sets of size up to d as our cog family optimization criteria. 

Since cogs from the same family define parity-check matrices of cyclic form with identical properties 
for the BEC and comparable properties for the AWGN channel, it is desirable to identify the family 
with the best performance and then exclusively use cogs from this family to build matrices for MBBP 
decoding. Identifying all families along with its members by counting stopping sets in the corresponding 
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matrices is computationally expensive [25]. Also, storing all cogs used for decoding can be prohibitively 
costly, especially for long codes and MBBP decoders with many bases. In order to avoid these problems, 
we propose to use a cog mapping technique that relies on the notion of the automorphism group of a 
code. 

Definition 3.1. [26, Ch. 8] The permutations which send C into itself, i.e. codewords go into (possibly 
different) codewords, form the automorphism group of the code C, denoted by Aut(C). If C is a linear 
code and C"*" is its dual code, then Aut(C) = Aut(C-'"). 

It is straightforward to prove that there exists a set of permutations V in the automorphism group 
of a cyclic code which map one cog into another cog from the same family. Fixing one cog, and then 
applying a subset of permutations from P to it, represents a convenient way for generating redundant 
parity-check matrices with identical densities and comparable performance under MBBP decoding. 

We provide next a partial characterization of the set V for cyclic codes. More precisely, we describe 
how to find a large set of permutations V for which \Scr{Ha)\ = \Scr{Hb)\, for a < n, where the 
generating cogs of Ha and Hf, satisfy cog;, = ^(coga), and where k(-) G V. Here, ^(coga) is used to 
denote the action of the permutation k on the vector coga. Note that the same framework can be used 
when only stopping sets up to a size smaller than or equal to d are considered. 

It is well known that the automorphism group of a cyclic code contains two classes of permuta- 
tions [26]: 

Pi: The class of cyclic permutations a^, . . . , a""^, where 

a : i + 1) modn, i = 0, . . . ,n — 1, 

a{c) = (c„_i, Co, Ci, . . . , c„_2). 

P2: The class of permutations . . . , where 

P : i — > (2 ■ i) modn, i = 0, . . . , n — 1, (2) 

f3{c) = (coC(„+i)/2Ci . . .C(„_i)/2), 

and where h denotes the cardinality of the cyclotomic coset of the n-th roots of unity that con- 
tains one. Above, all subscripts are taken modulo n. For extended cyclic codes, the described 
permutations are only to be applied to the cyclic part of the codeword. 

We find the following definition useful for our subsequent derivations. 

Definition 3.2. Let n be a permutation operating on a vector v, resulting in a vector t = k{v). We 
define the K-permutation of an m x n matrix V as a matrix obtained by permuting each row of V 
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according to k. In this setting, T = k{V) is used to denote T{fi, :) = K{V{fi, :)), fi = 1, . . . ,m, where 
X{fi,:) represents the fi-th row of the matrix X . 

The following straightforward results provide a partial characterization of the set of permutations V 
of cyclic codes. All proofs rely on the fact that 9 = 9 for any integer j and some integer j', and 
for 9 G A{n), the affine group of order n. 

Lemma 3.3. A necessary and sufficient condition for 

9{a^{coga)) = a^'{9{coga)), 

for allje{0,...,n- 1}, 
and some j' G {0, . . . , — 1}, 

to hold is that 9{-) is an affine permutation, defined as 

9 : i ^ q - i + uj modn, 

for some g G {0, . . . , n — 1}, c<j G {0, . . . , n — 1}, and i = 0, . . . , n — 1, 

such that gcd(g, n) = 1. Here, gcd(g, n) is used to denote the greatest common divisor of q and n. 
Proof. The claim of Lemma 3.3 can be rewritten as 

= 9{i) + j' modn 

J G {0, . . . , n - 1}, j' G {0, . . . , n - 1}, 2 = 0, . . . , n - 1, 

where 9{i) denotes the action of the permutation 9 on the coordinate i. 

The former equality is true if and only if 9{i) is a linear function of the form q ■ i + uj for which 
gcd(g, n) = 1. □ 

The lemma asserts that cyclic permutations commute (up to a cyclic shift) with all affine permuta- 
tions in a symmetric group. 

Example 3.1. Consider n = 5, j = 1, and 9 : i ^ i + 2 mod n, i = 0, . . . ,n — 1. Then, 0(a(lOlOO)) = 
a(6'(10100)) = 10010 holds. In this special case, both j and j' are equal to one. 

Claim 3.4. If cog b = a-' (cog a), for some j G {0, . . . , n — 1}, then 

\Sa {Ha)\ = iHb) \ , for all a <n. 

Proof. Applying to cog a cyclically permutes the rows of Ha- This cyclic permutation preserves all 
stopping sets, which proves the claimed result. □ 
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Claim 3.5. //cogc = P^{coga), for some j G {0, . . . , /i — 1}, then 

\S^{Ha)\ = \S^iH,)\, for all a <n. 

Proof. It is straiglitforward to see tliat 

since (3^ is an affine permutation witli q = 2 and u = 0. As a result, He can be transformed into Ha 
through row- and column-permutations. □ 

We conclude that A{n) fl Aut(C) C V: in other words, applying affine transforms from the auto- 
morphism group of the code to one chosen cog in the family produces cogs that generate parity-check 
matrices with identical stopping set distributions. Therefore, the MBBP decoder does not have to store 
all redundant bases, but rather a set of permutations, along with a low number of cog vectors that are 
known to have good stopping set properties. Note that this is a desirable property for practical applica- 
tions: in order to generate a set of matrices with almost identical and good decoding performance, one 
only needs to determine the stopping set properties of the cog families rather than that of all individual 
cogs. 

Let / denote the index of the optimal or near-optimal family. One vector cog i, I G and a subset 
of permutations in Aut(C) suffice to generate a set of cogs from JF^-. Depending on the set of available 
permutations, multiple cogs may be required to generate all cogs from the family J^j. With all these 
cogs at hand, one can construct the matrices H^, £ G J^j, required for MBBP decoding, by cyclically 
shifting the corresponding cogs, cf. Section 2. 

For some classes of cyclic codes, stronger results are available on the structure of the automorphism 
group. For example, it is known that the automorphism group of a primitive BCH code contains the 
affine group - and BCH cog families are invariant under the action of every element of the affine group. 
On the other hand, the family of binary extended quadratic residue (QR) codes [26] with lengths p+1, 
where p is a prime, is known to have automorphism groups that contain permutations from the projective 
special linear group PSL{p). For p = 8m ± 1, the elements of this group can be written as 

y '^^ a, b,c,dE F„, ad — be = 1. 
cy + d 

A non-affine permutation in this group transforms cogs from one family into cogs that belong to a 
different family. As a result, the image of the original cog generates a parity-check matrix with different 
stopping set properties compared to the one generated by the original cog. 
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4 MBBP decoding 



We describe next in more detail the operating principles of the MBBP decoders, and modifications 
thereof - as depicted in Figure 1 and 2, respectively. The basic components of an MBBP decoder are 
collections of (possibly redundant) parity-check matrices, and logical units that combine and process 
outputs of BP decoders operating on the matrices of the collection. We distinguish two basic MBBP 
architectures: one, which allows information exchange between decoders {MBBP-X) and another, where 
the decoder outputs are obtained without exchange of information (MBBP-NX). The decoders in the 
former category have the feature that information on the reliability of the received symbols can be 
exchanged during the process of iterative BP decoding; decoders in the latter class can only combine 
their results upon termination of their individual decoding processes. Both types of decoders can be 
implemented by storing a set of parity-check matrices. However, if only matrices based on one family of 
cogs are used, one needs to store only a low number of cogs, along with a set of permutations from V. 

The simplest architecture of an MBBP decoder is depicted in Figure 1, where the outputs of indi- 
vidual decoders are jointly processed only at the end of the decoding cycle. We refer to this technique 
as standard MBBP decoding (henceforth, MBBP-NX-S decoding). 

4.1 MBBP-NX decoding 

Standard MBBP decoding and its variation First-success MBBP decoding (MBBP-NX-FS) generate a 
collection of decoded words and then perform an additional metric selection within this set of words. 
The result of this processing is passed on to the information sink, which represents the gateway for the 
final codeword estimate of the decoder. 

4.1.1 Standard MBBP decoding (MBBP-NX-S) 

The MBBP-NX-S decoder runs multiple BP decoders in parallel, each of them separately and on a dif- 
ferent parity-check matrix representation of the code. Let the parity-check matrix representation used 
by the i-th decoder be denoted by H^, i = 1, . . . ,1, and its decoded vector after at most iterations by 
ce, i = 1, . . . ,1. We let V C {1, ... , /} be the set of indices i describing decoders that converged to a valid 
codeword. If none of the decoders converged to a valid codeword, we let V = {1, . . . , /}. The words esti- 
mated by the decoders, c„, w G V are passed on to a least metric selector (LMS) unit, which determines 
the "best" codeword estimate using the decision rule c = argmax Pr {Y = y \ C = q,}. It is well known 

n-l 

that for the AWGN channel, this equation can be rewritten [22] as c = argmin ^ ly^, — map(c^,^i,)| . 

In this context, map(-) defines the mapping of binary input symbols into antipodal signals (i.e. the 
BPSK modulation performed by the transmitter). The estimated information vector ii is obtained from 
c in the standard manner. We choose a generator matrix of systematic form what provides several 
advantages, described in more detail in a companion paper [27]. Figure 1 depicts the operation of the 
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MBBP-NX-S decoder. 
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Figure 1: MBBP-NX-S decoding 



4.1.2 First-success MBBP decoding (MBBP-NX-FS) 

Tliis type of MBBP decoder follows the standard approach in so far that it runs multiple BP decoders 
separately, each on a different parity-check matrix of the code. Denote the number of iterations required 
by the i-th decoder to converge by A^^. As soon as the first decoder, indexed by i*, identifies a codeword, 
the decoding process terminates. The estimate obtained by the decoder indexed by £* is passed on to 
the information sink. In the unlikely event that two or more decoders converge to a codeword after the 
same number of iterations, one of the outputs is randomly chosen from c^, where w G V and where V 
is defined as for the MBBP-NX-S decoder. This approach requires only A'^* = min^ Ni iterations to 
converge, and has therefore a significantly lower time-complexity when compared to MBBP-NX-S. When 
considering the average number of iterations to decode one codeword, this effect shows in particular in 
the low signal-to-noise ratio (SNR) regime. 

The steps of these algorithms are summarized in Algorithm 1. Note that the tags of a selected set 
of steps and commands in the table indicate the algorithm during which these steps are executed. All 
untagged steps are executed both during the MBBP-NX-S and the MBBP-NX-FS procedure. 

Line 2, 5, and 6 in Algorithm 1 describe the actions of detecting the first convergent decoder and 
terminating the decoding process. Although it can be easily shown that the MBBP-NX-FS algorithm 
cannot outperform its MBBP-NX-S counterpart, it provides for significant time savings, which makes 
it amenable for use in low-delay communication systems. 

4.2 MBBP-X decoding 

In this section we present MBBP approaches which allow for periodic exchange of information between 
decoders during iterative BP decoding. To this end, each decoder performs independently a given 
number of iterations, Np, and afterwards exchanges information with other decoders only at iterations 
indexed by l ■ Np, where i G N. The intrinsic information of a given variable node depends only on 
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Algorithm 1 MBBP-NX: Standard (NX-S), and First-Success (NX-FS) 
Input: y, Hi, .,Hi, N 
Output: c 

1: V := 0, i := 1 

2: while i < N (NX-S, NX-FS) and V = (NX-FS) do 

3: Ci := BPiteration(?/, Hi), £ G {1, . . . , /}\V 

4: for £:= 1,...,/ do 

5: if C£ ■ if J = then 

6: V:=VU£ 

7: end if 

8: end for 

9: i ■.= i + l 
10: end while 
11: if V = then 
12: V:={1,...,/} 
13: end if 

14: c := argmin ^ — map(ct,^,^) ^ (NX-S) 
15: c:= rand (NX-FS) 



the channel output and is therefore equal for all decoders. For this reason, the decoders exchange only 
extrinsic information about the variable nodes. Figure 2 depicts the basic architecture of an MBBP-X 
decoder. As before, ii can be obtained from c in a straightforward manner. 

To emphasize that the messages exchanged between decoders represent extrinsic information, they 
are denoted by Pr'''^''(-). As part of their cooperation scheme, the decoders agree on the (extrinsic) 
probability values for each variable node. Afterwards, each decoder replaces its own information about 
a given variable node with the jointly derived estimate of the decoders, then calculates the a-posteriori 
information, and resumes decoding. For the purpose of computing the cooperative extrinsic probability, 
only a subset of active decoders Au, v = 0, ... ,n — 1, is used. This subset is selected in terms of a 
soft-metric majority vote, which is described in more detail below. 

4.2.1 Probability- Averaging MBBP (MBBP-X-PA) 

This MBBP variation consists of the same basic steps as the standard MBBP algorithm, except for 
a periodic intra-decoder probability averaging feature. Probabilistic averaging was first described by 
the authors in [14], in the context of standard BP decoding. There, it was shown that it can lead to 
significant reductions in the error-fioor of the performance curve. Here, averaging is used in a different 
context: periodically, the parallel decoders update their extrinsic probabilities by setting them to the 
average probability of a subset of the best performing decoders. 

For this purpose, a soft-metric, weighted majority- vote is calculated for a variable node u according 

to 
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Figure 2: MBBP-X decoder architecture. The bus is active only iii = l ■ Np, l eN. 
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The subset of decoders used in the described averaging process is defined as 

'FT^''\ce,. = 0\He,y 



Au = { ^|sgn(i;^) = sgn ( log , 

Pr^ >{ce,u = 1 I He,y) 

where sgn(-) denotes the sign function. 

The averaged probabilities are determined as 
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(5) 



(6) 



We point out that the performance of this decoding approach strongly depends on the specific 
implementation of the proposed steps, as already pointed out in [14]. 

4.2.2 Highest-Reliability MBBP (MBBP-X-HR) 

This approach is a simple modification of the MBBP-X-PA technique, and it employs the weighted 
majority-vote introduced in Equation (3) to select the decoders used for computing the information 
update subsequently forwarded to all decoders. 

In this approach, the MBBP decoding architecture selects for each variable node an information 
update which is the most reliable output among all decoders, i.e. 



Pr(")(c^ = 0) = Pr(^)| 



I He*,y) 



(7) 
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with t = argmax \Pt^°' {ce/ = | Hei,y) — 0.5|, u = 0, . . . ,n — 1, and with A^, given according to 
Equation (4). Note that Pr^^-'(ci. = 1) is calculated according to Equation (6). 

4.2.3 Information-Combining MBBP (MBBP-X-IC) 

The optimal method for deciding on the value of a random variable when multiple independent noisy 
observations of a variable are given is information combining [15]. 

The extrinsic information provided by the parallel decoders depends on their underlying parity- 
check matrices and the received codeword y, and is hence not independent. Nevertheless, without 
assuming any optimality properties, we propose an MBBP decoding architecture which exchanges joint 
information about the variable nodes and determines this value by information combining. Information 
combining is also performed separately for each variable node as part of standard BP, and again only the 
decoders which agree with the soft-metric majority-vote according to Equation (3) are considered. In 
other words, only representations that have an index that belongs to the set A^, defined as in Equation 
(4), are used in this approach. 

The "combined" probability distribution of the binary variable X^, is given by 

n Pr(^)(c£',. = I 

Pr(^)(c, = 0) = n , z/ = 0,...,n- 1. (8) 

U Pr^''\c,,. = 0\He,y)+ U P^^''\c,,. = l\H,,y) 

Again, one can use Equation (6) to calculate Pr*-^-'(c;^ = 1). 

A summary of the steps for the three algorithms from the class of MBBP-X decoders is given in 
Algorithm 2. As before, the untagged lines are executed in all three algorithms, while the tags of the 
lines specify if the step is to be used in the X-PA, X-IC, or X-HR variation of MBBP decoding. 

5 Results 

As already pointed out, parity-check matrices that offer good performance when used for decoding signals 
transmitted over the BEC may also be good candidates for BP decoding over the AWGN channel [28]. 
The same characteristic of parity-check matrices was observed through extensive computer simulations 
for the variants of MBBP decoders, used over the AWGN channel. 

In this section, we present performance results for MBBP decoders. Whenever possible, we provide 
a comparison of the error rates of MBBP decoders with those of a full search algorithm (maximum- 
likelihood, ML, decoding). If performing full search is computationally infeasible, but results on the 
weight distribution {Aj}, i = 0, 1, . . . , n, of the underlying code are available, we plot the union bounds 
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Algorithm 2 MBBP-X: Probability- Averaging (PA), Information-Combining (IC), and Highest- 
Reliability (HR) 
Input: y, Hi,..., Hi, N, Np 
Output: c 

1 
2 
3 
4 
5 
6 
7; 



9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 



V := 0, i := 1 

Pr(°)(c^=0|/f^,y) = 0.5, u = 0,...,n-l,e=l,...,l 
while i < N and V = do 

[Q,Pr('=)(Q,, = I He,y)] := BPiteration(y, if,, Pr(°)(c,=0|/f^, 2/)), z/ = 0, . . . , 



if i = L ■ Np, L e N then 
Apply Eq. (3) 
Apply Eq. (4) ^ A 

Apply Eqs. (5) and (6) ^ Pr(")(c^ = 0/1), u = 0,...,n-l (X-PA) 
Apply Eq. (7)^ Pr(^)(c^ = 0/1), u = 0,...,n-l (X-HR) 
Apply Eq. (8)^ Pr^^^fc^ = 0/1), u = 0,...,n-l (X-IC) 
Pr^'^) (c,=0 1 H,, y):=Fi^''^ (c, = 0), u = 0, . . . ,n - 1, £ = 1, . . . ,1 
for £ := 1, . . . , / do 
if Q ■ HJ = then 

V:=VUi 
end if 
end for 
end if 
i:=i+l 
end while 
if V = then 

V:={1,...,/} 
end if 
c := rand 



for the BER and PER as given by the expressions below: 

Here, it is tacitly assumed that the transmitted codeword is the all-zero word, and Ai is used to denote 
the number of codewords of weight i in the given code. Furthermore, we compare our results to the 
Gallager bound (random coding bound) [29]. This bound considers an ensemble of block codes with 
length n and rate R and provides a tight upper bound on the average PER, denoted by E{FER}. In 
this context, the expectation is taken over the ensemble of codes. According to [29], the upper bound 
reads 
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E{FER} < exp -n ■ max max (Eo(p, 10 logio(Eb/iVo), Pr{X)) - pR) ] , (9) 

\ 0<p<l Pr(X) J 

where p, < p < 1, is a design parameter and Pr(X) denotes the probabihty vector of the possible 
channel inputs. Let us specialize this bound for the AWGN channel, which has discrete inputs and an 
SNR-dependent transition density function fy[y \ Xi). Here, Xi is chosen from a finite set of cardinality 
Mx and y is a continuous variable, and thus 

Eo(p,101ogio(^b/iVo),Pr(X)) = - In 

We illustrate our findings on four short-to-moderate length codes; these codes belong to the class of 
cyclic and extended cyclic codes, and they include the [24,12,8] extended Golay code, the [47,24,11] 
quadratic residue (QR) code, the [31, 16, 7]-BCH code, as well as the [127, 64, 21]-BCH code. The 
maximum number of iterations is set to = 100 for all codes and decoding approaches considered. 
Furthermore, Np is set to 10 whenever MBBP-X approaches are simulated. 

Throughout the remainder of this section, / is used to denote the number of parallel BP decoders 
in the MBBP architecture. For comparison, simulation results for standard BP decoding are presented 
as well. Also shown are the performance results of a BP decoder using the union of all parity-check 
equations in ii"^, £ = 1, . . . , Z, simultaneously, for the case of the [31, 16, 7]-BCH code. 

5.1 The [24,12,8] Extended Golay Code 

For the purpose of MBBP decoding of the extended Golay code, we use the result of Sections 2 and 
3, and identify three different cog families denoted by JF^, JF2, and JF3. For this, and all subsequently 
considered codes, one can generate cogs of a family by repeated application of permutations of type P2, 
shown in Equation (2). As an example, we will describe the process of constructing the parity-check 
matrices for the extended Golay code in more detail. 

Since the code is an extended cyclic code, we construct the parity-check matrices from each cog in 
terms of 23 shifts performed on positions to 22 while keeping the last position fixed. For the 24-th 
row of the parity-check matrix, we use the all-one codeword: this codeword preserves the stopping set 
distribution of the 23 x 24 matrix, and is the only parity check of the [24, 12, 8] extended Golay code 
invariant under all affine permutations. It is worth pointing out that there are other choices for the 24-th 
parity-check equation that may lead to slightly better overall performance - we use this parity-check 
matrix for simplicity of analysis. As a performance criterion for the cog families, we use the number of 
stopping sets up to size d = 8 in the parity-check matrices H^, i G {^1,^2,^3}- 

We consider one representative cog for each family, termed cog^c-^, / = 1, 2, 3. A list of these cogs is 
given below. 




Pr(xi) 



dy 
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= 6 


437 


46 


a 


= 7 


1357 10143 


1495 


a 


= 8 


25783 73209 


20631 



Table 1: Number of stopping sets for the [24,12,8] extended Golay code, with parity-check matrices 
Hi, coge G J^/, / = 1, . . . ,3. 



110101001100100000001000 
111000001001100000100001 
110100110000000101001000 

Each cogjF^,, / G {1,2,3}, allows for generating all cogs in the family jFj by repeated apphcation 
of permutation P2 to the first 23 positions of cogjc-^, while keeping the last position fixed, cf. Equation 
(2). The permutation P2 is an automorphism of an extended cyclic code and preserves the stopping 
set distribution of the matrices Hi, i G J-'f, cf. Claim 3.5. The number of stopping sets in Hi, 
i G {JF]^, JF2, JF3}, is summarized in Table 1. Note that the matrices of the considered families differ 
significantly from the near-optimal and highly redundant matrices used for decoding over the BEC, 
given in [6] and [7] . Due to the high redundancy, these matrices are not amenable for decoding over the 
AWGN channel, where short cycles may significantly degrade the performance of iterative decoders. 

The [24,12,8] extended Golay code is self-dual and contains 759 codewords of minimum weight 8 
which can be partitioned into 33 cyclic orbits. Again, these cyclic orbits can be partitioned into three 
families of equal size. In other words, each family jFj, / = 1, 2, 3, contains 11 cogs. 

If matrices from the same family are used for signaling over the BEC, they provide the same per- 
formance under iterative decoding. Interestingly, the simulation results presented below show that the 
same is true for decoders used over the AWGN channel. 

Figure 3 shows the BER performance of MBBP-NX-S decoding and MBBP-NX-FS decoding as well 
as MBBP-X-PA, MBBP-X-HR, and MBBP-X-IC of the extended Golay code. For all decoder types, 
/ = 11 parallel BP decoders are used, for which the parity-check matrices are all drawn from the same 
family. FER performance results show similar characteristics. These results are not plotted due to space 
limitations. 

It can be observed for all approaches that MBBP decoders using matrices Hi, I E T\, have the 
best performance, followed by matrices Hi, £ G JF3. This finding is supported by the stopping set 
distribution of Table 1, showing that the first family contains matrices that do not have stopping sets 
of size up to six and a low number of stopping sets of size seven. Matrices from the other two families 
have stopping sets of size six and exceed the number of stopping sets of size seven of the first family. 



cog^i = 
cog ^2 = 
cog ^3 = 
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T 



10" 



10 : 



10" 




3 3.5 4 4.5 5 5.5 



3 3.5 4 4.5 5 5.5 




(b) BER performance of MBBP-X approaches 
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Figure 3: Performance comparison for the [24, 12, 8] extended Golay code using Hi, I G J^2i -^3- 

The performance of MBBP decoders using parity-check matrices i/^, I G J^2i is significantly worse than 
that of the two other classes - matrices in this family have 437 stopping sets of size six and over 10, 000 
stopping sets of size seven. 

The performance obtained by means of the MBBP-NX-S with / = 11 is approximately 0.75 dB better 
than standard BP and performs close to the ML decoding bound. When performing a direct comparison 
of MBBP-NX-S and MBBP-NX-FS decoders, cf. Figure 3, one can observe that MBBP-NX-FS follows 
the performance of MBBP-NX-S very closely. However, simulation results show a significant gap in the 
average number of iterations required for successful decoding when comparing these approaches. As the 
shapes of these curves show similar characteristics, and due to space limitations, these results are only 
plotted for the [127, 64, 21]-BCH code. It was observed for all codes discussed within this paper that 
MBBP-NX-FS requires significantly fewer iterations per decoder than MBBP-NX-S and BP decoding. 

It is worth pointing out that the MBBP-X-PA and MBBP-X-HR algorithms produce similar results 
for all possible choices of cog families. Also, if the cogs are chosen from JF^ or JF3, the MBBP-X ap- 
proaches outperform standard BP decoding, but do not attain the performance of MBBP- NX decoders. 
If the decoders operate on parity-check matrices constructed from cogs in JF2, very poor performance 
results and error floors are observed in most of the cases. MBBP-X-IC decoders perform very poorly, 
regardless of the family considered. A probable cause for this phenomena is the strong correlation of 
the data, which makes information combining techniques highly suboptimal. 



The dual of the [31, 16, 7]-BCH code, denoted by iSj*", has minimum Hamming distance equal to eight. 
The minimum-weight codewords in Bi can be partitioned into 15 cyclic orbits. The corresponding cogs 
belong to one single family - as a result, all cog-based parity-check matrices of cyclic form with the same 



5.2 [31,16,7]-BCH code 
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number of rows are expected to have comparable performance. In this special case we require 3 cogs, 
all from the same family, to generate the 15 possible cogs. A list of these cogs is given below. 



cog^,,i = 11110000100110000000000000 
cog^,,2 = 11000011000001010000000000 
cog^,,3 = 11100000000100101000010010 



Figure 4 visualizes the performance results for MBBP decoding of the [31, 16, 7]-BCH code. We used 
/ = 6 decoders for the MBBP-NX, MBBP-X-PA, MBBP-X-HR, and MBBP-X-IC algorithms. 
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Figure 4: Performance comparison for the [31, 16, 7]-BCH code. 



It can be observed that the MBBP-X techniques and the MBBP-NX-FS decoders have inferior per- 
formance compared to the MBBP-NX-S approach. Yet, these approaches require fewer iterations when 
more decoders run in parallel, i.e. they usually operate with lower complexity. It is worth mentioning 
that the NX-FS approach exhibits better performance than standard BP decoding and requires fewer 
iterations for successful decoding. 

The performance curve labeled "stacked" corresponds to a BP decoder operating on a parity-check 
matrix which contains the union of parity checks present in Hg, £ = 1,...,6. Observe that this 
performance is significantly worse even when compared to a standard BP decoder running on H^, 
£ G {1, . . . , 6}. Reasons for this include the local cycle distribution of the stacked matrix. All matrices 
Hg, i = 1, . . . , 6, have a large number of short cycles, and the stacked matrix has an even larger number 
of such cycles, and is therefore not a good candidate for BP decoding. 
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5.3 [47,24,11]-QR code 

We present next MBBP simulation results for the [47,24, 11]-QR code, henceforth denoted by Q. The 
code Q"*" has minimum Hamming distance 12, and there exist 276 codewords of this weight. These 
codewords can be partitioned into eight cog famihes, labeled J-'i to jFg. In order to identify the family 
with best stopping set properties, we compute the number of stopping sets of size up to and including 
cr = 9 in cog-based matrices of each family. Table 2 allows for identifying the family with the best 
stopping set properties, JF3. This family contains 23 cogs. Figure 5 plots simulation results for / = 23 
representations with cogs chosen from JF3. 
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Table 2: Number of stopping sets for parity-check matrices of cyclic form for the [47, 24, 11]-QR code. 



- Standard BP decoding 

- MBBP-NX-S, I = 23 

- MBBP-NX-FS, / = 23 
MBBP-X-PA, I = 23 
MBBP-X-IC, / = 23 
MBBP-X-HR, I = 23 
ML Union bound 
Gallager bound 
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Figure 5: Performance comparison for the [47, 24, 11]-QR code. 



We list the results for the [47, 24, 11]-QR code as these results allow to point out two remarkable 
properties of MBBP decoding. First, we observe that MBBP decoding can provide a decoding perfor- 
mance which is superior to the performance reported by the Gallager bound. This is remarkable, as the 
Gallager bound indicates desirable performance results for codes of given length and rate. Compared to 
this bound, standard BP decoding obtains by far poorer performance results. Second, the [47,24, 11]- 
QR code shows that non-standard MBBP decoders can outperform their standard counterparts. The 
approaches MBBP-X-PA and MBBP-X-HR lead to performance resuhs close to MBBP-NX-S, while the 
approach MBBP-X-PA even outperforms MBBP-NX-FS. As in the previous examples, the MBBP-X-IC 
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decoder suffers from the introduced dependencies that arise due to data exchange between the decoders 
via information combining. For this reason, the latter approach shows a slightly degraded performance, 
yet still significantly better than that of standard BP. 



5.4 [127, 64, 21]-BCH code 

The [127, 64, 21]-BCH code, denoted as B2, has dimension 64 and a co-dimension 63. It is prohibitively 
costly to decode this code by means of an ML decoder or a trellis decoder. Also, it is computationally 
infeasible to find all minimum weight codewords of the dual code. Instead, we apply the algorithms 
provided in [30] to identify a tight upper bound on the minimum distance of the dual code as well as 
a (most likely incomplete) set of codewords with weight equal to the estimated minimum distance of 
the dual code. We report that 22 is an upper bound on the minimum distance of B2, and that at least 
21 cog families exist. As it is a very complex task to identify the number of stopping sets up to the 
size (i = 21 in a code of length 127, we approximate the performance criterion by evaluating only the 
number of stopping sets up to a = 5. This allows us to conclude that all families yield comparable 
decoding performance. 

Figure 6 shows the BER performance for different variants of the MBBP approach, where / = 10 
decoders were run in parallel. There exist similar observations for the FER, but these results are not 
shown due to space limitations. Also given is the number of iterations required for the convergence of 
different decoding approaches. In this context, we distinguish between the average number of iterations 
required and the maximum number of iterations required to decode one codeword. It is also worth 
pointing out that these results are representative for all codes discussed in this work. 




101ogio(^b/iVo) ^ 101ogip(^b/iVo) -> 

(a) BER performance (b) Average and maximum number of iterations 

Figure 6: Performance comparison and number of iterations for the [127, 64, 21]-BCII code. 



We observe a behavior that is characteristic for MBBP-NX decoding in connection with codes of 
high dimension and co-dimension. A significant performance improvement compared to BP decoding 
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is obtained. However, there remains a gap to the Gallager bound. For this particular code, we observe 
that MBBP-NX-FS is a very promising approach. In this case, a significant decrease in the number 
of average iterations can be obtained by using the MBBP-NX-FS decoder, while keeping the decoding 
performance close to that of the MBBP-NX-S class. However, all MBBP-X approaches perform very 
poorly. Reasons for this phenomena include the fact that, due to communication between the decoders, 
the number of "virtual short cycles" may be increased. 

The performance of the MBBP-NX-S decoder matches the performance obtained with the ordered 
statistics decoding [2] for first order processing. The approach in [2] requires for each codeword a 
sorting of the received vector according to the received symbol reliability. Furthermore, a Gauss- 
Jordan algorithm is applied on the correspondingly resorted generator matrix, and, in order to obtain 
a decoding performance comparable to MBBP-NX-S with / = 10, up to 64 pattern tests are performed. 
We emphasize that MBBP avoids the use of Gauss- Jordan's algorithm, which is the computationally 
most costly component of the decoder in [2] . Raising the number of test patterns further, the decoder 
in [2] can outperform the MBBP-NX-S approach with I = 10 parallel decoders. Similar observations 
are made for the Chase Type-2 algorithm. While MBBP-NX-S outperforms Chase Type-2 decoders 
for codes of short length [31], Chase Type-2 is superior for longer codes such as the [127, 64, 21]-BCH 
code [32]. This is strongly related to the number of test patterns, which reads 2^^^'^^ = 1024 and hence 
significantly higher than the diversity order in any reasonable MBBP-NX-S setup. 

6 Conclusions 

We introduced a class of decoding algorithms that operate in parallel on a judiciously chosen family of 
parity-check matrices. We considered two variants of this class of techniques: one, in which the decoders 
are not allowed to exchange information during individual runs of the BP algorithm, and another, in 
which periodic information exchange is allowed. Algorithms in the first class were shown to offer 
significant performance improvements when compared to the standard BP technique. The approaches 
in the second class often compare favorably to standard BP, but do not match the performance of 
algorithms that do not make use of periodic information exchange. Possible reasons for this behavior 
include the fact that "cycles" are created during the process of information exchange. These cycles 
"in-between" the graphs of the representations negatively affect the performance of each decoder. 

The presented approaches were shown to work for classical high-density codes and are applicable to 
cyclic and extended cyclic codes. It is possible to generalize the introduced methods to codes like the 
progressive edge-growth (PEG) [33] family. For this class of codes, significant gains in performance can 
be obtained [34]. 



21 



References 



[1] Y. Han, C. Hartmann, and C. Chen, "Efficient maximum-likelihood soft-decision decoding of lin- 
ear block codes using algorithm A*," in Proceedings of the IEEE International Symposium on 
Information Theory (ISIT), (San Antonio, Texas, USA), p. 27, January 1993. 

[2] M. Fossorier and S. Lin, "Soft-decision decoding of linear block codes based on ordered statistics," 
IEEE Transactions on Information Theory, vol. 41, pp. 1379-1396, September 1995. 

[3] R. Lucas, M. Bossert, and M. Breitbach, "On iterative soft-decision decoding of linear binary 
block codes and product codes," IEEE Journal on Selected Areas in Communications, vol. 16, 
pp. 276-296, February 1998. 

[4] T. Halford and K. Chugg, "Random redundant soft-in soft-out decoding of linear block codes," in 
Proceedings of the IEEE International Symposium on Information Theory (ISIT), (Seattle, Wash- 
ington, USA), pp. 2230-2234, July 2006. 

[5] T. Halford, A. Grant, and K. Chugg, "Which codes have 4-cycle-free Tanner graphs?," IEEE 
Transactions on Information Theory, vol. 52, pp. 4219-4223, September 2006. 

[6] M. Schwartz and A. Vardy, "On the stopping distance and stopping redundancy of codes," IEEE 
Transactions on Information Theory, vol. 52, pp. 922-932, March 2006. 

[7] J. Han and P. Siegel, "Improved upper bounds on stopping redundancy," IEEE Transactions on 
Information Theory, vol. 53, pp. 90-104, January 2007. 

[8] K. Abdel-Ghaffar and J. Weber, "Complete enumeration of stopping sets of full-rank parity-check 
matrices of Hamming codes," IEEE Transactions on Information Theory, vol. 53, pp. 3196-3201, 
September 2007. 

[9] H. Hollmann and L. Tolhuizen, "On parity-check collections for iterative erasure decoding that 
correct all correctable erasure patterns of a given size," IEEE Transactions on Information Theory, 
vol. 53, pp. 823-828, February 2007. 

[10] A. Kothiyal, O. Y. Takeshita, W. Jin, and M. Fossorier, "Iterative reliability-based decoding of lin- 
ear block codes with adaptive belief propagation," IEEE Communications Letters, vol. 9, pp. 1067- 
1069, December 2005. 

[11] J. Jiang and K. Narayanan, "Iterative soft-input soft-output decoding of Reed-Solomon codes by 
adapting the parity-check matrix," IEEE Transactions on Information Theory, vol. 52, pp. 3746- 
3756, August 2006. 

[12] K. Andrews, S. Dolinar, and F. Pollara, "LDPC decoding using multiple representations," in Pro- 
ceedings of the IEEE International Symposium on Information Theory (ISIT), (Lausanne, Switzer- 
land), p. 456, June 2002. 



22 



[13] T. Hehn, O. Milenkovic, S. Laendner, and J. Huber, "Permutation decoding and the stopping 
redundancy hierarchy of cychc and extended cychc codes," IEEE Transactions on Information 
Theory, vol 54, pp. 5308-5331, December 2008. 

[14] S. Laendner and O. Milenkovic, "Algorithmic and combinatorial analysis of trapping sets in struc- 
tured LDPC codes," in Proceedings of the International Conference on Wireless Networks, Com- 
munications, and Mobile Computing (WirelessComm), (Maui, Hawaii), pp. 630-635, June 2005. 

[15] I. Land and J. Huber, "Information combining," Foundations and Trends in Communications and 
Information Theory, vol. 3, pp. 227-330, November 2006. 

[16] L. Zeng, L. Lan, Y. Y. Tai, S. Lin, and K. Abdel-Ghaffar, "Construction of LDPC codes for 
AWGN and binary erasure channels based on finite fields," in Proceedings of IEEE Information 
Theory Workshop (ITW), (Rotorua, New Zealand), pp. 273-276, August 2005. 

[17] C. Di, D. Proietti, I. Telatar, T. Richardson, and R. Urbanke, "Finite-length analysis of low- 
density parity-check codes on the binary erasure channel," IEEE Transactions on Information 
Theory, vol. 48, pp. 1570-1579, June 2002. 

[18] J. Feldman, Decoding Error- Correcting Codes via Linear Programming. PhD thesis, Massachusetts 
Institute of Technology, Cambridge, MA, USA, 2003. 

[19] C. Kelley and D. Sridhara, "Pseudocodewords of Tanner graphs," IEEE Transactions on Informa- 
tion Theory, vol. 53, pp. 4013-4038, November 2007. 

[20] R. Koetter and P. Vontobel, "Graph covers and iterative decoding of finite-length codes," in Pro- 
ceedings of the 3rd International Symposium on Turbo Codes & Related Topics, (Brest, France), 
pp. 75-82, September 2003. 

[21] J. Kovacevic and A. Chebira, "Life beyond bases: The advent of frames (Part I)," IEEE Signal 
Processing Magazine, vol. 24, pp. 86 - 104, July 2007. 

[22] S. Lin and D. Costello, Error Control Coding. Pearson Education, Inc., second ed., 2004. 

[23] S. Laendner, T. Hehn, O. Milenkovic, and J. Huber, "When does one redundant parity-check 
equation matter?," in Proceedings of the 49th annual IEEE Global Telecommunications Conference 
(GlobeCom), (San Francisco, USA), November 2006. 

[24] T. Hehn, S. Laendner, O. Milenkovic, and J. B. Huber, "The stopping redundancy hierarchy of 
cyclic codes," in Proceedings of the 44th Annual Allerton Conference on Communication, Control 
and Computing, (Monticello, USA), pp. 1271-1280, September 2006. 

[25] A. McGregor and O. Milenkovic, "On the hardness of approximating stopping and trapping sets 
in LDPC codes," in Proceedings of the IEEE Information Theory Workshop (ITW), (Lake Tahoe, 
Cahfornia, USA), pp. 248-253, September 2007. 



23 



[26] F. MacWilliams and N. Sloane, The Theory of Error- Correcting Codes. Amsterdam, The Nether- 
lands: North-Holland Publishing Company, 1977. 

[27] T. Hehn and J. Huber, "LDPC codes and convolutional with equal structural delay: A comparison," 
Accepted for publication in IEEE Transactions on Communications, September 2008. 

[28] T. Richardson, M. Shokrollahi, and R. Urbanke, "Design of capacity-approaching irregular low- 
density parity-check codes," IEEE Transactions on Information Theory, vol. 47, pp. 619-637, 
February 2001. 

[29] R. G. Gallager, Information Theory and Reliable Communication. John Wiley and Sons, 1968. 

[30] X.-Y. Hu, M. Fossorier, and E. Eleftheriou, "On the computation of the minimum distance of 
low-density parity-check codes," in Proceedings of the IEEE International Conference on Commu- 
nications (ICC), (Paris, France), pp. 767-771, June 2004. 

[31] T. Hehn, J. Huber, S. Laendner, and O. Milenkovic, "Multiple-bases belief-propagation decoding 
for short block-codes," in Proceedings of the IEEE International Symposium on Information Theory 
(ISIT), (Nice, France), pp. 311-315, June 2007. 

[32] M. Fossorier and S. Lin, "Chase-type and GMD coset decodings," IEEE Transactions on Commu- 
nications, vol. 48, pp. 345-350, March 2000. 

[33] X.-Y. Hu, E. Eleftheriou, and D. M. Arnold, "Regular and irregular progressive edge-growth Tanner 
graphs," IEEE Transactions on Information Theory, vol. 51, pp. 386-398, January 2005. 

[34] T. Hehn, Optimized Belief-Propagation Decoding for Low-Delay Applications in Digital Communi- 
cations. PhD thesis. University of Erlangen-Nuremberg, Germany, 2009. 



24 



